System and methods to validate issue detection and classification in a network assurance system

ABSTRACT

Systems, methods and computer-readable storage media are provided for detecting and simulating issues in a network.

CROSS-REFERENCE

This application is related to and claims priority under 35 U.S.C. §119(e) from U.S. Patent Appl. No. 62/770,279, filed Nov. 21, 2018entitled “SYSTEMS AND METHODS TO VALIDATE ISSUE DETECTION ANDCLASSIFICATION IN A NETWORK ASSURANCE SYSTEM,” the entire contents ofeach of which is incorporated herein by reference for all purposes.

TECHNICAL FIELD

The subject matter of this disclosure relates in general to the field ofcomputer networking, and more particularly, to systems and methods forimproving the operation of an enterprise network.

BACKGROUND

A campus network can provide connectivity to computing devices (e.g.,servers, workstations, desktop computers, laptop computers, tablets,mobile phones, etc.) and things (e.g., desk phones, security cameras,lighting, heating, ventilating, and air-conditioning (HVAC), windows,doors, locks, medical devices, industrial and manufacturing equipment,etc.) within environments such as offices, hospitals, colleges anduniversities, oil and gas facilities, factories, and similar locations.Some of the unique challenges a campus network may face includeintegrating wired and wireless devices, on-boarding computing devicesand things that can appear anywhere in the network and maintainingconnectivity when the devices and things migrate from location tolocation within the network, supporting bring your own device (BYOD)capabilities, connecting and powering Internet-of-Things (IoT) devices,and securing the network despite the vulnerabilities associated withWi-Fi access, device mobility, BYOD, and IoT.

Current approaches for deploying a network capable of providing thesefunctions often require constant and extensive configuration andadministration by highly skilled network engineers operating severaldifferent systems (e.g., directory-based identity services;authentication, authorization, and accounting (AAA) services, wirelesslocal area network (WLAN) controllers; command line interfaces for eachswitch, router, or other network device of the network; etc.) andmanually stitching these systems together. This can make networkdeployment difficult and time-consuming, and impede the ability of manyorganizations to innovate rapidly and adopt new technologies, such asvideo, collaboration, and connected workspaces.

BRIEF DESCRIPTION OF THE FIGURES

To provide a more complete understanding of the present disclosure andfeatures and advantages thereof, reference is made to the followingdescription, taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 illustrates an example of a physical topology of an enterprisenetwork in accordance with some examples;

FIG. 2 illustrates an example of a logical architecture for anenterprise network in accordance with some examples;

FIGS. 3A-3I illustrate examples of graphical user interfaces for anetwork management system in accordance with some examples;

FIG. 4 illustrates an example of a physical topology for a multi-siteenterprise network in accordance with some examples;

FIG. 5A illustrates an example of an issue validation system for anetwork management system in accordance with some examples;

FIG. 5B illustrates an example classification engine for a networkmanagement system in according with some examples;

FIGS. 5Ci-5Cii illustrate transaction event sequences in accordance withsome examples;

FIG. 5D illustrates transaction event issue analysis graphs inaccordance with some examples;

FIG. 5E illustrates a histogram of excess transaction event patterns inaccordance with some examples;

FIG. 5F illustrates a frequency table for excess transaction eventpatterns in accordance with some examples;

FIG. 5G illustrates a method for identifying and validating issues for anetwork management system in accordance with some examples; and

FIGS. 6A and 6B illustrate examples of systems in accordance with someexamples.

DESCRIPTION OF EXAMPLE EMBODIMENTS

The detailed description set forth below is intended as a description ofvarious configurations of embodiments and is not intended to representthe only configurations in which the subject matter of this disclosurecan be practiced. The appended drawings are incorporated herein andconstitute a part of the detailed description. The detailed descriptionincludes specific details for the purpose of providing a more thoroughunderstanding of the subject matter of this disclosure. However, it willbe clear and apparent that the subject matter of this disclosure is notlimited to the specific details set forth herein and may be practicedwithout these details. In some instances, structures and components areshown in block diagram form in order to avoid obscuring the concepts ofthe subject matter of this disclosure.

Overview

Systems, methods and computer-readable storage media for issuevalidation by performing issue simulation and reproduction for detectedissues are described. In some cases, a data collection service capturesdata models of raw streaming telemetry data from network devices as itis passed to a network management and assurance service. The telemetrydata can include client event data such as access statistics, packetcaptures, device key performance indicators (KPIs), service KPIs, andthe like. Sequences of client events are observed and recorded in orderto generate a simplified definition of issues defined in terms of eventpatterns and associated with, for example and without imputinglimitation, failure, roaming, and normal onboarding patterns. Using thesimplified definitions, issues can be simulated by an issue simulationengine to validate issues detected by the network management andassurance service. In addition, the simplified definitions can beprocessed by a machine learning engine to perform clustering and issueprediction in order to increase the accuracy and speed of issueidentification.

In at least some aspects, an example system, method, orcomputer-readable storage medium can receive network traffic data,identify event sequences within the network traffic data, generatedefinition groups by performing clustering on the identified eventsequences, the definition groups including event sequencecharacteristics associated with one or more network issues, provide oneor more of the event sequences to an issue simulation engine to simulatethe one or more event sequences and generating a first issueidentification by an assurance platform, generate a second issueidentification based on the definition groups and the one or more of theevent sequences; and validate the first issue identification with thesecond issue identification.

Example Embodiments

Network assurance platforms may be a single point in a networkmanagement system where network issues may be detected, classified, anddisplayed to a customer. However, there is often no way to verify thatall issues have been detected by the assurance platform or that thedetected issues are being properly classified.

An external (to the assurance platform) and independent source of truthcan verify and validate issue spotting by the assurance platform. Insome cases, raw telemetry data from a Wireless Local Area NetworkController (WLC), packet captures, or Technical Assurance Centers(TAC)/manual assurance can provide verification and validation of theassurance platform. However, these solutions may fail to handle manykinds of client behavior or may not be unable to handle issues that areoutside the design parameters of a particular solution (e.g., it maylack the ability to identify issues that occur in customer sites buthave not been well defined and integrated into a respective detectionsystem).

In other words, a system is needed for independent issue detection andclassification, cataloging issues in order to check whether an issue isnew or already defined, and automated reproduction of issues for anassurance system to validate whether the issue was detected andclassified correctly (e.g., providing reproducibility). An example of asystem for detecting and classifying issues, cataloging issues, andreproducing and/or validating issues, can include five high level steps.These steps may include collecting data, identifying event patterns forissue identification, simulating issues, executing an assurance platformon the simulated issues, and validating results from the assuranceplatform.

Data Collection

Network devices may export client streaming telemetry data to anassurance and/or analytics engine. The network devices can include, forexample and without imputing limitation, Wireless LAN controllers (WLC)and/or Access Points (AP). Data models of the raw streaming telemetrydata from WLCs and APs may be attached to the streaming telemetry data.The data models can include, for example, wireless client state, clientand AP radio frequency (RF) statistics, over air packet captures,network services KPIs, device health KPIs, etc. for the purposes of thisdisclosure, these are called client events.

Issue Identification and Definition Engine

Client events, such as those sent by the WLC and AP, can be identifiedaccording to various sequences and patterns. The client events may bebased off real client behavior. The sequences can form a specificpattern that may indicate and/or define issues with specific focus onfailure, roaming, and normal onboarding patterns. Variations mayinclude, for example and without imputing limitation, changes in SSID,radios, and APs, timing differences between client events, retries ofthe client events, and reason codes, etc.

As a result, a simplified definition of issues may be created which canbe catalogued and used to replay events to generate issues by anassurance and/or analytics platform. In order to identify and defineissues, the following steps can be undergone:

(a) collect WLC and AP client event data;

(b) analyze all data in order to extract patterns of onboardingsequences;

(c) tag the sequences with category identifiers;

(d) represent each of the sequences with client events in a single linemodel; and

(e) simplify the single line model by a cluster mechanism to identifythe repetitive components.

A learning engine using the above data identifies new issues andclusters. The learning engine may use semi-supervised learning. Forexample, a first round of unsupervised clustering may be performed andthen a second round of human/expert clustering may be performed tocreate a catalog of issues.

The learning engine can catalog and cluster issues in the form of ameta-language. For example, when wireless clients run into onboardingfailures, a simple representation of the onboarding failure issue couldbe in the form of: {timestamp, event-type, result-type, reason-code,packet-capture}. The meta-language form of the onboarding failure issuecan then be leveraged by a test generator to generate synthetic tests(discussed below).

Issue Simulation

The issued identified above can be reproduced by the WLC, AP, or asimulation service. The recreated real world client scenarios may thenbe used to generate issue identifications (e.g., by the networkassurance and/or analytics platform) for verification, experimentation,and the like.

The simulation service can automatically generate synthetic tests fromthe issue catalogue using a meta-language. The synthetic tests can beprovided to the WLCs and/or Aps or, in some examples, respectivesimulations in order to verify whether the issue is detected andclassified accurately (e.g., by an assurance platform, probabilisticclassifier, etc.). Further, the data collected by the learning enginefrom different sources at different time windows may help to generatesynthetic tests that can be executed at different network devices.

Continuing with the example of onboarding failures of wireless clients,the test generator can generate a sequence of events based on the{timestamp, event-type, result-type, reason-code, packet-capture}meta-data captured in the issue catalog and feed it into the WLCs and/orthe APs to trigger issue detection in the assurance and/or analyticsplatform. This may help in verifying if issues are being correctlyidentified and categorized.

Assurance and Analytics Platform Issue Generation

The assurance and analytics platform may receive incoming data andtrigger an issue when conditions are satisfied according to its ownrules and definitions. For example, an unmodified assurance andanalytics platform may receive incoming data produced by the simulationservice and generate issue detections and solution options as if thedata were real world events being produced by WLCs and APs withoutdirection of a simulation engine.

Issue Validation

As issues are generated by the assurance and/or analytics platform, thegenerated issues can be confirmed by the issue identification enginewhen conditions are satisfied.

In comparison, in typical cases not using the above approach, issuevalidation is often based on an expert determining what issues to testand then using a synthetic test to validate the system by unit orregression testing. Instead, the above approach can learn of issues(e.g., failure patterns) independently and based on customer data. Thelearned issues can then be used to drive validation and/or updating ofthe assurance and/or analytics platform. Additionally, characterizingissues characterized in a meta-data form enables cataloging and curatingissues and respective underlying event sequence information (e.g., forlater reproduction, analysis, experiment, etc.).

For example, patterns can be processed to identify an “excessivetransaction.” As discussed in this disclosure, an excessive transactionis a transaction in which a client joins a wireless network in anunreasonable amount of time. A transaction is a series of client eventsfor a given client and may start with, for example, an AssociationRequest (e.g., “ASSOC_START”). Transactions may end with either a Runevent or any other event with a result-type of failure. Examples ofclient events may include, for example and without imputing limitation,‘ASSOC_START’, ‘ASSOC_DONE’, ‘MOBILITY_DONE’, ‘RUN’, ‘DHCP_DONE’,‘AUTH_START’, ‘EAPOL_START’, ‘AUTH_DONE’, ‘BROADCAST_REKEY’,‘SEND_DEAUTH’, ‘DELETE’, ‘L2KEY_DONE’, ‘SEND_DISASSOC’, ‘RECV_DISASSPC’,‘RECV_DEAUTH’, and ‘WEB AUTH_DONE’.

In particular, where a successful transaction takes more than 10seconds, it can be identified as an excess onboarding transaction due tothe amount of time taken. Excess transactions may often end in a Runstate. However, excess transactions take an extended amount of time toresolve relative to a normal amount of time (e.g., average transactiontime, etc.) for the respective transaction and, in many cases, one ormore event sequences can be identified as relating to an excesstransaction. For example, a large number of ASSOC_START events andAUTH_START events may be observed in an event sequence related to anexcess transaction. Certain sequences may account for more excesstransactions than others. In some examples, the top 10 sequences, orpatterns, may account for 50% of all excess transactions even though avery large number of possible patterns may exist.

In effect, onboarding issues can be identified by: filtering outsequences of events observed associated with a client failure; filteringout corresponding event-reason codes; and then tracking additionalcontext such as changes in radio, SSID during the sequence, etc. Issuegroups based on common characteristics of the above information can thenbe created. For example, 60,000 failure events may be grouped intoapproximately 1,000 groups on a first clustering pass. Groups withdistinct event patterns may be aggregated into a single group based oncommon characteristics such as consecutive AUTH_START events and/orAUTH_DONE events. As a result, customer data can be used to discover newissues that would be otherwise undetected and unimplemented intoassurance platforms. Further, assurance and/or analytics platforms maybe validated for issue identification and classification.

Intent-based networking is an approach for overcoming the deficiencies,discussed above and elsewhere in the present disclosure, of conventionalenterprise networks. The motivation of intent-based networking is toenable a user to describe in plain language what he or she wants toaccomplish (e.g., the user's intent) and have the network translate theuser's objective into configuration and policy changes that areautomatically propagated across a complex and heterogeneous computingenvironment. Thus, an intent-based network can abstract networkcomplexity, automate much of the work of provisioning and managing thenetwork typically handled by a network administrator, and assure secureoperation and optimal performance of the network. As an intent-basednetwork becomes aware of the users, devices, and things makingconnections in the network, it can automatically apply securitypermissions and service levels in accordance with the privileges andquality of experience (QoE) assigned to the users, devices, and things.Table 1 sets forth examples of intents and workflows that can beautomated by an intent-based network to achieve a desired outcome.

TABLE 1 Examples of Intents and Associated Workflows Intent Workflow Ineed to scale out my Extend network segments; update load balancerapplication database configuration; configure quality of service (QoS) Ihave scheduled a Create high-definition (HD) video connection;telemedicine session prioritize with end-to-end QoS; validate at 10amperformance; keep the communication safe; tear down connection aftercall I am rolling out a new Create a new segment for all factory devicesto IoT app for factory connect to the IoT app; isolate from othertraffic; equipment monitoring apply service level agreement (SLA);validate SLA; optimize traffic flow I need to deploy a Provisionmultiple networks and subnets; secure multi-tier configure accesscontrol lists (ACLs) and firewall application rules; advertise routinginformation

Some additional examples of use cases of an intent-based network:

-   -   An intent-based network can learn the performance needs of        applications and services and adapt the network from end-to-end        to achieve specified service levels;    -   Instead of sending technicians to every office, floor, building,        or branch, an intent-based network can discover and identify        devices and things as they connect, assign security and        micro-segmentation profiles according to established policies,        and continuously monitor access point performance to        automatically adjust for QoE;    -   Users can move freely among network segments, mobile device in        hand, and automatically connect with the correct security and        access privileges;    -   Switches, routers, and other network devices can be powered up        by local non-technical office personnel, and the network devices        can be configured remotely (by a user or by the network) via a        cloud management console with the appropriate policies as        defined by the intents for the specific location (e.g.,        permanent employee access, visiting employee access, guest        access, etc.); and    -   Machine learning and artificial intelligence agents running in        the network can continuously monitor and analyze network traffic        and connections, compare activity against pre-defined intents        such as application performance or security policies, detect        malware intrusions in encrypted traffic and automatically        isolate infected devices, and provide a historical record of        network events for analysis and troubleshooting.

FIG. 1 illustrates an example of a physical topology of an enterprisenetwork 100 for providing intent-based networking. It should beunderstood that, for the enterprise network 100 and any networkdiscussed herein, there can be additional or fewer nodes, devices,links, networks, or components in similar or alternative configurations.Example embodiments with different numbers and/or types of endpoints,nodes, cloud components, servers, software components, devices, virtualor physical resources, configurations, topologies, services, appliances,or deployments are also contemplated herein. Further, the enterprisenetwork 100 can include any number or type of resources, which can beaccessed and utilized by endpoints or network devices. The illustrationsand examples provided herein are for clarity and simplicity.

In this example, the enterprise network 100 includes a management cloud102 and a network fabric 120. Although shown as an external network orcloud to the network fabric 120 in this example, the management cloud102 may alternatively or additionally reside on the premises of anorganization or in a colocation center (in addition to being hosted by acloud provider or similar environment). The management cloud 102 canprovide a central management plane for building and operating thenetwork fabric 120. The management cloud 102 can be responsible forforwarding configuration and policy distribution, as well as devicemanagement and analytics. The management cloud 102 can comprise one ormore network controller appliances 104, one or more authentication,authorization, and accounting (AAA) appliances 106, one or more wirelesslocal area network controllers (WLCs) 108, and one or more fabriccontrol plane nodes 110. In other embodiments, one or more elements ofthe management cloud 102 may be co-located with the network fabric 120.

The network controller appliance(s) 104 can function as the command andcontrol system for one or more network fabrics, and can house automatedworkflows for deploying and managing the network fabric(s). The networkcontroller appliance(s) 104 can include automation, design, policy,provisioning, and assurance capabilities, among others, as discussedfurther below with respect to FIG. 2. In some embodiments, one or moreCisco Digital Network Architecture (Cisco DNA™) appliances can operateas the network controller appliance(s) 104.

The AAA appliance(s) 106 can control access to computing resources,facilitate enforcement of network policies, audit usage, and provideinformation necessary to bill for services. The AAA appliance caninteract with the network controller appliance(s) 104 and with databasesand directories containing information for users, devices, things,policies, billing, and similar information to provide authentication,authorization, and accounting services. In some embodiments, the AAAappliance(s) 106 can utilize Remote Authentication Dial-In User Service(RADIUS) or Diameter to communicate with devices and applications. Insome embodiments, one or more Cisco® Identity Services Engine (ISE)appliances can operate as the AAA appliance(s) 106.

The WLC(s) 108 can support fabric-enabled access points attached to thenetwork fabric 120, handling traditional tasks associated with a WLC aswell as interactions with the fabric control plane for wireless endpointregistration and roaming. In some embodiments, the network fabric 120can implement a wireless deployment that moves data-plane termination(e.g., VXLAN) from a centralized location (e.g., with previous overlayControl and Provisioning of Wireless Access Points (CAPWAP) deployments)to an access point/fabric edge node. This can enable distributedforwarding and distributed policy application for wireless traffic whileretaining the benefits of centralized provisioning and administration.In some embodiments, one or more Cisco® Wireless Controllers, Cisco®Wireless LAN, and/or other Cisco DNA™-ready wireless controllers canoperate as the WLC(s) 108.

The network fabric 120 can comprise fabric border nodes 122A and 122B(collectively, 122), fabric intermediate nodes 124A-D (collectively,124), and fabric edge nodes 126A-F (collectively, 126). Although thefabric control plane node(s) 110 are shown to be external to the networkfabric 120 in this example, in other embodiments, the fabric controlplane node(s) 110 may be co-located with the network fabric 120. Inembodiments where the fabric control plane node(s) 110 are co-locatedwith the network fabric 120, the fabric control plane node(s) 110 maycomprise a dedicated node or set of nodes or the functionality of thefabric control node(s) 110 may be implemented by the fabric border nodes122.

The fabric control plane node(s) 110 can serve as a central database fortracking all users, devices, and things as they attach to the networkfabric 120, and as they roam around. The fabric control plane node(s)110 can allow network infrastructure (e.g., switches, routers, WLCs,etc.) to query the database to determine the locations of users,devices, and things attached to the fabric instead of using a flood andlearn mechanism. In this manner, the fabric control plane node(s) 110can operate as a single source of truth about where every endpointattached to the network fabric 120 is located at any point in time. Inaddition to tracking specific endpoints (e.g., /32 address for IPv4,/128 address for IPv6, etc.), the fabric control plane node(s) 110 canalso track larger summarized routers (e.g., IP/mask). This flexibilitycan help in summarization across fabric sites and improve overallscalability.

The fabric border nodes 122 can connect the network fabric 120 totraditional Layer 3 networks (e.g., non-fabric networks) or to differentfabric sites. The fabric border nodes 122 can also translate context(e.g., user, device, or thing mapping and identity) from one fabric siteto another fabric site or to a traditional network. When theencapsulation is the same across different fabric sites, the translationof fabric context is generally mapped 1:1. The fabric border nodes 122can also exchange reachability and policy information with fabriccontrol plane nodes of different fabric sites. The fabric border nodes122 also provide border functions for internal networks and externalnetworks. Internal borders can advertise a defined set of known subnets,such as those leading to a group of branch sites or to a data center.External borders, on the other hand, can advertise unknown destinations(e.g., to the Internet similar in operation to the function of a defaultroute).

The fabric intermediate nodes 124 can operate as pure Layer 3 forwardersthat connect the fabric border nodes 122 to the fabric edge nodes 126and provide the Layer 3 underlay for fabric overlay traffic.

The fabric edge nodes 126 can connect endpoints to the network fabric120 and can encapsulate/decapsulate and forward traffic from theseendpoints to and from the network fabric. The fabric edge nodes 126 mayoperate at the perimeter of the network fabric 120 and can be the firstpoints for attachment of users, devices, and things and theimplementation of policy. In some embodiments, the network fabric 120can also include fabric extended nodes (not shown) for attachingdownstream non-fabric Layer 2 network devices to the network fabric 120and thereby extend the network fabric. For example, extended nodes canbe small switches (e.g., compact switch, industrial Ethernet switch,building automation switch, etc.) which connect to the fabric edge nodesvia Layer 2. Devices or things connected to the fabric extended nodescan use the fabric edge nodes 126 for communication to outside subnets.

In this example, the network fabric can represent a single fabric sitedeployment which can be differentiated from a multi-site fabricdeployment as discussed further below with respect to FIG. 4.

In some embodiments, all subnets hosted in a fabric site can beprovisioned across every fabric edge node 126 in that fabric site. Forexample, if the subnet 10.10.10.0/24 is provisioned in a given fabricsite, this subnet may be defined across all of the fabric edge nodes 126in that fabric site, and endpoints located in that subnet can be placedon any fabric edge node 126 in that fabric. This can simplify IP addressmanagement and allow deployment of fewer but larger subnets. In someembodiments, one or more Cisco® Catalyst switches, Cisco Nexus®switches, Cisco Meraki® MS switches, Cisco® Integrated Services Routers(ISRs), Cisco® Aggregation Services Routers (ASRs), Cisco® EnterpriseNetwork Compute Systems (ENCS), Cisco® Cloud Service Virtual Routers(CSRvs), Cisco Integrated Services Virtual Routers (ISRvs), CiscoMeraki® MX appliances, and/or other Cisco DNA-Ready™ devices can operateas the fabric nodes 122, 124, and 126.

The enterprise network 100 can also include wired endpoints 130A, 130C,130D, and 130F and wireless endpoints 130B and 130E (collectively, 130).The wired endpoints 130A, 130C, 130D, and 130F can connect by wire tofabric edge nodes 126A, 126C, 126D, and 126F, respectively, and thewireless endpoints 130B and 130E can connect wirelessly to wirelessaccess points 128B and 128E (collectively, 128), respectively, which inturn can connect by wire to fabric edge nodes 126B and 126E,respectively. In some embodiments, Cisco Aironet® access points, CiscoMeraki® MR access points, and/or other Cisco DNA™-ready access pointscan operate as the wireless access points 128.

The endpoints 130 can include general purpose computing devices (e.g.,servers, workstations, desktop computers, etc.), mobile computingdevices (e.g., laptops, tablets, mobile phones, etc.), wearable devices(e.g., watches, glasses or other head-mounted displays (HMDs), eardevices, etc.), and so forth. The endpoints 130 can also includeInternet of Things (IoT) devices or equipment, such as agriculturalequipment (e.g., livestock tracking and management systems, wateringdevices, unmanned aerial vehicles (UAVs), etc.); connected cars andother vehicles; smart home sensors and devices (e.g., alarm systems,security cameras, lighting, appliances, media players, HVAC equipment,utility meters, windows, automatic doors, door bells, locks, etc.);office equipment (e.g., desktop phones, copiers, fax machines, etc.);healthcare devices (e.g., pacemakers, biometric sensors, medicalequipment, etc.); industrial equipment (e.g., robots, factory machinery,construction equipment, industrial sensors, etc.); retail equipment(e.g., vending machines, point of sale (POS) devices, Radio FrequencyIdentification (RFID) tags, etc.); smart city devices (e.g., streetlamps, parking meters, waste management sensors, etc.); transportationand logistical equipment (e.g., turnstiles, rental car trackers,navigational devices, inventory monitors, etc.); and so forth.

In some embodiments, the network fabric 120 can support wired andwireless access as part of a single integrated infrastructure such thatconnectivity, mobility, and policy enforcement behavior are similar orthe same for both wired and wireless endpoints. This can bring a unifiedexperience for users, devices, and things that is independent of theaccess media.

In integrated wired and wireless deployments, control plane integrationcan be achieved with the WLC(s) 108 notifying the fabric control planenode(s) 110 of joins, roams, and disconnects by the wireless endpoints130 such that the fabric control plane node(s) can have connectivityinformation about both wired and wireless endpoints in the networkfabric 120, and can serve as the single source of truth for endpointsconnected to the network fabric. For data plane integration, the WLC(s)108 can instruct the fabric wireless access points 128 to form a VXLANoverlay tunnel to their adjacent fabric edge nodes 126. The AP VXLANtunnel can carry segmentation and policy information to and from thefabric edge nodes 126, allowing connectivity and functionality identicalor similar to that of a wired endpoint. When the wireless endpoints 130join the network fabric 120 via the fabric wireless access points 128,the WLC(s) 108 can onboard the endpoints into the network fabric 120 andinform the fabric control plane node(s) 110 of the endpoints' MediaAccess Control (MAC) addresses. The WLC(s) 108 can then instruct thefabric wireless access points 128 to form VXLAN overlay tunnels to theadjacent fabric edge nodes 126. Next, the wireless endpoints 130 canobtain IP addresses for themselves via Dynamic Host ConfigurationProtocol (DHCP). Once that completes, the fabric edge nodes 126 canregister the IP addresses of the wireless endpoint 130 to the fabriccontrol plane node(s) 110 to form a mapping between the endpoints' MACand IP addresses, and traffic to and from the wireless endpoints 130 canbegin to flow.

FIG. 2 illustrates an example of a logical architecture 200 for anenterprise network (e.g., the enterprise network 100). One of ordinaryskill in the art will understand that, for the logical architecture 200and any system discussed in the present disclosure, there can beadditional or fewer component in similar or alternative configurations.The illustrations and examples provided in the present disclosure arefor conciseness and clarity. Other embodiments may include differentnumbers and/or types of elements but one of ordinary skill the art willappreciate that such variations do not depart from the scope of thepresent disclosure. In this example, the logical architecture 200includes a management layer 202, a controller layer 220, a network layer230 (such as embodied by the network fabric 120), a physical layer 240(such as embodied by the various elements of FIG. 1), and a sharedservices layer 250.

The management layer 202 can abstract the complexities and dependenciesof other layers and provide a user with tools and workflows to manage anenterprise network (e.g., the enterprise network 100). The managementlayer 202 can include a user interface 204, design functions 206, policyfunctions 208, provisioning functions 210, assurance functions 212,platform functions 214, and base automation functions 216. The userinterface 204 can provide a user a single point to manage and automatethe network. The user interface 204 can be implemented within a webapplication/web server accessible by a web browser and/or anapplication/application server accessible by a desktop application, amobile app, a shell program or other command line interface (CLI), anApplication Programming Interface (e.g., restful state transfer (REST),Simple Object Access Protocol (SOAP), Service Oriented Architecture(SOA), etc.), and/or other suitable interface in which the user canconfigure network infrastructure, devices, and things that arecloud-managed; provide user preferences; specify policies, enter data;review statistics; configure interactions or operations; and so forth.The user interface 204 may also provide visibility information, such asviews of a network, network infrastructure, computing devices, andthings. For example, the user interface 204 can provide a view of thestatus or conditions of the network, the operations taking place,services, performance, a topology or layout, protocols implemented,running processes, errors, notifications, alerts, network structure,ongoing communications, data analysis, and so forth.

The design functions 206 can include tools and workflows for managingsite profiles, maps and floor plans, network settings, and IP addressmanagement, among others. The policy functions 208 can include tools andworkflows for defining and managing network policies. The provisioningfunctions 210 can include tools and workflows for deploying the network.The assurance functions 212 can use machine learning and analytics toprovide end-to-end visibility of the network by learning from thenetwork infrastructure, endpoints, and other contextual sources ofinformation. The platform functions 214 can include tools and workflowsfor integrating the network management system with other technologies.The base automation functions 216 can include tools and workflows tosupport the policy functions 208, the provisioning functions 210, theassurance functions 212, and the platform functions 214.

In some embodiments, the design functions 206, the policy functions 208,the provisioning functions 210, the assurance functions 212, theplatform functions 214, and the base automation functions 216 can beimplemented as microservices in which respective software functions areimplemented in multiple containers communicating with each rather thanamalgamating all tools and workflows into a single software binary. Eachof the design functions 206, policy functions 208, provisioningfunctions 210, assurance functions 212, and platform functions 214 canbe viewed as a set of related automation microservices to cover thedesign, policy authoring, provisioning, assurance, and cross-platformintegration phases of the network lifecycle. The base automationfunctions 214 can support the top-level functions by allowing users toperform certain network-wide tasks.

FIGS. 3A-3I illustrate examples of graphical user interfaces forimplementing the user interface 204. Although FIGS. 3A-3I show thegraphical user interfaces as comprising webpages displayed in a browserexecuting on a large form-factor general purpose computing device (e.g.,server, workstation, desktop, laptop, etc.), the principles disclosed inthe present disclosure are applicable to client devices of other formfactors, including tablet computers, smart phones, wearable devices, orother small form-factor general purpose computing devices; televisions;set top boxes; IoT devices; and other electronic devices capable ofconnecting to a network and including input/output components to enablea user to interact with a network management system. One of ordinaryskill will also understand that the graphical user interfaces of FIGS.3A-3I are but one example of a user interface for managing a network.Other embodiments may include a fewer number or a greater number ofelements.

FIG. 3A illustrates a graphical user interface 300A, which is an exampleof a landing screen or a home screen of the user interface 204. Thegraphical user interface 300A can include user interface elements forselecting the design functions 206, the policy functions 208, theprovisioning functions 210, the assurance functions 212, and theplatform functions 214. The graphical user interface 300A also includesuser interface elements for selecting the base automation functions 216.In this example, the base automation functions 216 include:

-   -   A network discovery tool 302 for automating the discovery of        existing network elements to populate into inventory;    -   An inventory management tool 304 for managing the set of        physical and virtual network elements;    -   A topology tool 306 for visualizing the physical topology of        network elements;    -   An image repository tool 308 for managing software images for        network elements;    -   A command runner tool 310 for diagnosing one or more network        elements based on a CLI;    -   A license manager tool 312 for administering visualizing        software license usage in the network;    -   A template editor tool 314 for creating and authoring CLI        templates associated with network elements in a design profile;    -   A network PnP tool 316 for supporting the automated        configuration of network elements;    -   A telemetry tool 318 for designing a telemetry profile and        applying the telemetry profile to network elements; and    -   A data set and reports tool 320 for accessing various data sets,        scheduling data extracts, and generating reports in multiple        formats (e.g., Post Document Format (PDF), comma-separate values        (CSV), Tableau, etc.), such as an inventory data report, a        software image management (SWIM) server report, and a client        data report, among others.

FIG. 3B illustrates a graphical user interface 300B, an example of alanding screen for the design functions 206. The graphical userinterface 300B can include user interface elements for various tools andworkflows for logically defining an enterprise network. In this example,the design tools and workflows include:

-   -   A network hierarchy tool 322 for setting up the geographic        location, building, and floor plane details, and associating        these with a unique site id;    -   A network settings tool 324 for setting up network servers        (e.g., Domain Name System (DNS), DHCP, AAA, etc.), device        credentials, IP address pools, service provider profiles (e.g.,        QoS classes for a WAN provider), and wireless settings;    -   An image management tool 326 for managing software images and/or        maintenance updates, setting version compliance, and downloading        and deploying images;    -   A network profiles tool 328 for defining LAN, WAN, and WLAN        connection profiles (including Service Set Identifiers (SSIDs));        and    -   An authentication template tool 330 for defining modes of        authentication (e.g., closed authentication, Easy Connect, open        authentication, etc.).

The output of the design workflow 206 can include a hierarchical set ofunique site identifiers that define the global and forwardingconfiguration parameters of the various sites of the network. Theprovisioning functions 210 may use the site identifiers to deploy thenetwork.

FIG. 3C illustrates a graphical user interface 300C, an example of alanding screen for the policy functions 208. The graphical userinterface 300C can include various tools and workflows for definingnetwork policies. In this example, the policy design tools and workflowsinclude:

-   -   A policy dashboard 332 for viewing virtual networks, group-based        access control policies, IP-based access control policies,        traffic copy policies, scalable groups, and IP network groups.        The policy dashboard 332 can also show the number of policies        that have failed to deploy. The policy dashboard 332 can provide        a list of policies and the following information about each        policy: policy name, policy type, policy version (e.g.,        iteration of policy which can be incremented each time the        policy changes, user who has modified the policy, description,        policy scope (e.g., user and device groups or applications that        the policy affects), and timestamp;    -   A group-based access control policies tool 334 for managing        group-based access controls or SGACLs. A group-based access        control policy can define scalable groups and an access contract        (e.g., rules that make up the access control policies, such as        permit or deny when traffic matches on the policy);    -   An IP-based access control policies tool 336 for managing        IP-based access control policies. An IP-based access control can        define an IP network group (e.g., IP subnets that share same        access control requirements) and an access contract;    -   An application policies tool 338 for configuring QoS for        application traffic. An application policy can define        application sets (e.g., sets of applications that with similar        network traffic needs) and a site scope (e.g., the site to which        an application policy is defined);    -   A traffic copy policies tool 340 for setting up an Encapsulated        Remote Switched Port Analyzer (ERSPAN) configuration such that        network traffic flow between two entities is copied to a        specified destination for monitoring or troubleshooting. A        traffic copy policy can define the source and destination of the        traffic flow to copy and a traffic copy contract that specifies        the device and interface where the copy of traffic is sent; and    -   A virtual network policies tool 343 for segmenting the physical        network into multiple logical networks.

The output of the policy workflow 208 can include a set of virtualnetworks, security groups, and access and traffic policies that definethe policy configuration parameters of the various sites of the network.The provisioning functions 210 may use the virtual networks, groups, andpolicies for deployment in the network.

FIG. 3D illustrates a graphical user interface 300D, an example of alanding screen for the provisioning functions 210. The graphical userinterface 300D can include various tools and workflows for deploying thenetwork. In this example, the provisioning tools and workflows include:

-   -   A device provisioning tool 344 for assigning devices to the        inventory and deploying the required settings and policies, and        adding devices to sites; and    -   A fabric provisioning tool 346 for creating fabric domains and        adding devices to the fabric.

The output of the provisioning workflow 210 can include the deploymentof the network underlay and fabric overlay, as well as policies (definedin the policy workflow 208).

FIG. 3E illustrates a graphical user interface 300E, an example of alanding screen for the assurance functions 212. The graphical userinterface 300E can include various tools and workflows for managing thenetwork. In this example, the assurance tools and workflows include:

-   -   A health overview tool 344 for providing a global view of the        enterprise network, including network infrastructure devices and        endpoints. The user interface element (e.g., drop-down menu, a        dialog box, etc.) associated with the health overview tool 344        can also be toggled to switch to additional or alternative        views, such as a view of the health of network infrastructure        devices alone, a view of the health of all wired and wireless        clients, and a view of the health of applications running in the        network as discussed further below with respect to FIGS. 3F-3H;    -   An assurance dashboard tool 346 for managing and creating custom        dashboards;    -   An issues tool 348 for displaying and troubleshooting network        issues; and    -   A sensor management tool 350 for managing sensor-driven tests.

The graphical user interface 300E can also include a location selectionuser interface element 352, a time period selection user interfaceelement 354, and a view type user interface element 356. The locationselection user interface element 354 can enable a user to view theoverall health of specific sites (e.g., as defined via the networkhierarchy tool 322) and/or network domains (e.g., LAN, WLAN, WAN, datacenter, etc.). The time period selection user interface element 356 canenable display of the overall health of the network over specific timeperiods (e.g., last 3 hours, last 24 hours, last 7 days, custom, etc.).The view type user interface element 355 can enable a user to togglebetween a geographical map view of the sites of the network (not shown)or a hierarchical site/building view (as shown).

Within the hierarchical site/building view, rows can represent thenetwork hierarchy (e.g. sites and buildings as defined by the networkhierarchy tool 322); column 358 can indicate the number of healthyclients as a percentage; column 360 can indicate the health of wirelessclients by a score (e.g., 1-10), color and/or descriptor (e.g., red orcritical associated with a health score 1 to 3 indicating the clientshave critical issues, orange or warning associated with a health scoreof 4 to 7 indicating warnings for the clients, green or no errors orwarnings associated with a health score of 8 to 10, grey or no dataavailable associated with a health score of null or 0), or otherindicator; column 362 can indicate the health of wired clients by score,color, descriptor, and so forth; column 364 can include user interfaceelements for drilling down to the health of the clients associated witha hierarchical site/building; column 366 can indicate the number ofhealthy network infrastructure devices as a percentage; column 368 canindicate the health of access switches by score, color, descriptor, andso forth; column 370 can indicate the health of core switches by score,color, descriptor, and so forth; column 372 can indicate the health ofdistribution switches by score, color, descriptor, and so forth; column374 can indicate the health of routers by score, color, descriptor, andso forth; column 376 can indicate the health of WLCs by score, color,descriptor, and so forth; column 378 can indicate the health of othernetwork infrastructure devices by score, color, descriptor, and soforth; and column 380 can include user interface elements for drillingdown to the health of the network infrastructure devices associated witha hierarchical site/building. In other embodiments, client devices maybe grouped in other ways besides wired or wireless, such as by devicetype (e.g., desktop, laptop, mobile phone, IoT device or more specifictype of IoT device, etc.), manufacturer, model, operating system, and soforth. Likewise, network infrastructure devices can also be groupedalong these and other ways in additional embodiments.

The graphical user interface 300E can also include an overall healthsummary user interface element (e.g., a view, pane, tile, card,container, widget, dashlet, etc.) that includes a client health summaryuser interface element 384 indicating the number of healthy clients as apercentage, a color coded trend chart 386 indicating that percentageover a specific time period (e.g., as selected by the time periodselection user interface element 354), a user interface element 388breaking down the number of healthy clients as a percentage by clienttype (e.g., wireless, wired), a network infrastructure health summaryuser interface element 390 indicating the number of health networkinfrastructure devices as a percentage, a color coded trend chart 392indicating that percentage over a specific time period, and a userinterface element 394 breaking down the number of network infrastructuredevices as a percentage by network infrastructure device type (e.g.,core switch, access switch, distribution switch, etc.).

The graphical user interface 300E can also include an issues userinterface element 396 listing issues, if any, that must be addressed.Issues can be sorted based on timestamp, severity, location, devicetype, and so forth. Each issue may be selected to drill down to view amore detailed view of the selected issue.

FIG. 3F illustrates a graphical user interface 300F, an example of ascreen for an overview of the health of network infrastructure devicesalone, which may be navigated to, for instance, by toggling the healthoverview tool 344. The graphical user interface 300F can include atimeline slider 398 for selecting a more granular time range than a timeperiod selection user interface element (e.g., the time period selectionuser interface element 354). The graphical user interface 300F can alsoinclude similar information to that shown in the graphical userinterface 300E, such as a user interface element comprising ahierarchical site/building view and/or geographical map view similar tothat of the graphical user interface 300E (except providing informationonly for network infrastructure devices) (not shown here), the number ofhealthy network infrastructure devices as a percentage 390, the colorcoded trend charts 392 indicating that percentage by device type, thebreakdown of the number of healthy network infrastructure devices bydevice type 394, and so forth. In addition, the graphical user interface300F can display a view of the health of network infrastructure devicesby network topology (not shown). This view can be interactive, such asby enabling a user to zoom in or out, pan left or right, or rotate thetopology (e.g., by 90 degrees).

In this example, the graphical user interface 300F also includes a colorcoded trend chart 3002 showing the performance of the networkinfrastructure devices over a specific time period; network health bydevice type tabs including a system health chart 3004 providing systemmonitoring metrics (e.g., CPU utilization, memory utilization,temperature, etc.), a data plane connectivity chart 3006 providing dataplane metrics, such as uplink availability and link errors, and acontrol plane connectivity chart 3008 providing control plane metricsfor each device type; an AP analytics user interface element includingan up and down color coded chart 3010 that provides AP statusinformation (e.g., the number of APs connected to the network, and thenumber of APs not connected to the network, etc.) and a top number N ofAPs by client count chart 3012 that provides information about the APsthat have the highest number of clients; a network devices table 3014enabling a user to filter (e.g., by device type, health, or customfilters), view, and export network device information. A detailed viewof the health of each network infrastructure device can also be providedby selecting that network infrastructure device in the network devicestable 3014.

FIG. 3G illustrates a graphical user interface 300G, an example of ascreen for an overview of the health of client devices, which may benavigated to, for instance, by toggling the health overview tool 344.The graphical user interface 300G can include an SSID user interfaceselection element 3016 for viewing the health of wireless clients by allSSIDs or a specific SSID, a band frequency user interface selectionelement 3018 for viewing the health of wireless clients by all bandfrequencies or a specific band frequency (e.g., 2.4 GHz, 5 GHz, etc.),and a time slider 3020 that may operate similarly to the time slider398.

The graphical user interface 300G can also include a client healthsummary user interface element that provides similar information to thatshown in the graphical user interface 300E, such as the number ofhealthy clients as a percentage 384 and a color coded trend chart 386indicating that percentage over a specific time period for each groupingof client devices (e.g., wired/wireless, device type, manufacturer,model, operating system, etc.). In addition, the client health summaryuser interface element can include a color-coded donut chart thatprovides a count of poor (e.g., red and indicating a client health scoreof 1 to 3), fair (e.g., orange and indicating a client health score of 4to 7), good (e.g., green and indicating a health score of 8 to 10), andinactive (e.g., grey and indicating a health score that is null or 0)client devices. The count of client devices associated with each color,health score, health descriptor, and so forth may be displayed by aselection gesture directed toward that color (e.g., tap, double tap,long press, hover, click, right-click, etc.).

The graphical user interface 300G can also include a number of otherclient health metric charts in all sites or a selected site over aspecific time period, such as:

-   -   Client onboarding times 3024;    -   Received Signal Strength Indications (RSSIs) 3026;    -   Connectivity signal-to-noise ratios (SNRs) 3028;    -   Client counts per SSID 3030;    -   Client counts per band frequency 3032;    -   DNS requests and response counters (not shown); and    -   Connectivity physical link state information 3034 indicating the        distribution of wired client devices that had their physical        links up, down, and had errors.

In addition, the graphical user interface 300G can include a clientdevices table 3036 enabling a user to filter (e.g., by device type,health, data (e.g., onboarding time>threshold, associationtime>threshold, DHCP>threshold, AAA>threshold, RSSI>threshold, etc.), orcustom filters), view, and export client device information (e.g., useridentifier, hostname, MAC address, IP address, device type, last heard,location, VLAN identifier, SSID, overall health score, onboarding score,connection score, network infrastructure device to which the clientdevice is connected, etc.). A detailed view of the health of each clientdevice can also be provided by selecting that client device in theclient devices table 3036.

FIG. 3H illustrates a graphical user interface 300H, an example of ascreen for an overview of the health of applications, which may benavigated to, for instance, by the toggling the health overview tool344. The graphical user interface 300H can include application healthsummary user interface element including a percentage 3038 of the numberof healthy applications as a percentage, a health score 3040 for eachapplication or type of application (e.g., business relevant, businessirrelevant, default; HTTP, VoIP, chat, email, bulk transfer,multimedia/streaming, etc.) running in the network, a top number N ofapplications by usage chart 3042. The health score 3040 can becalculated based on an application's qualitative metrics, such as packetloss, network latency, and so forth.

In addition, the graphical user interface 300H can also include anapplications table 3044 enabling a user to filter (e.g., by applicationname, domain name, health, usage, average throughput, traffic class,packet loss, network latency, application latency, custom filters,etc.), view, and export application information. A detailed view of thehealth of each application can also be provided by selecting thatapplication in the applications table 3044.

FIG. 3I illustrates an example of a graphical user interface 300I, anexample of a landing screen for the platform functions 210. Thegraphical user interface 300C can include various tools and workflowsfor integrating with other technology systems. In this example, theplatform integration tools and workflows include:

-   -   A bundles tool 3046 for managing packages of domain-specific        APIs, workflows, and other features for network programming and        platform integration;    -   A developer toolkit 3048 for accessing an API catalog listing        the available APIs and methods (e.g., GET, PUT, POST, DELETE,        etc.), descriptions, runtime parameters, return codes, model        schemas, and so forth. In some embodiments, the developer        toolkit 3048 can also include a “Try It” button to permit a        developer to experiment with a particular API to better        understand its behavior;    -   A runtime dashboard 3050 for viewing and analyzing basic metrics        or API and integration flow usage;    -   A platform settings tool 3052 to view and set global or        bundle-specific settings that define integration destinations        and event consumption preferences; and    -   A notifications user interface element 3054 for presenting        notifications regarding the availability of software updates,        security threats, and so forth.

Returning to FIG. 2, the controller layer 220 can comprise subsystemsfor the management layer 220 and may include a network control platform222, a network data platform 224, and AAA services 226. These controllersubsystems can form an abstraction layer to hide the complexities anddependencies of managing many network elements and protocols.

The network control platform 222 can provide automation andorchestration services for the network layer 230 and the physical layer240, and can include the settings, protocols, and tables to automatemanagement of the network and physical layers. For example, the networkcontrol platform 230 can provide the design functions 206, theprovisioning functions 208 212. In addition, the network controlplatform 230 can include tools and workflows for discovering switches,routers, wireless controllers, and other network infrastructure devices(e.g., the network discovery tool 302); maintaining network and endpointdetails, configurations, and software versions (e.g., the inventorymanagement tool 304); Plug-and-Play (PnP) for automating deployment ofnetwork infrastructure (e.g., the network PnP tool 316), Path Trace forcreating visual data paths to accelerate the troubleshooting ofconnectivity problems, Easy QoS for automating quality of service toprioritize applications across the network, and Enterprise ServiceAutomation (ESA) for automating deployment of physical and virtualnetwork services, among others. The network control platform 222 cancommunicate with network elements using Network Configuration(NETCONF)/Yet Another Next Generation (YANG), Simple Network ManagementProtocol (SNMP), Secure Shell (SSH)/Telnet, and so forth. In someembodiments, the Cisco® Network Control Platform (NCP) can operate asthe network control platform 222

The network data platform 224 can provide for network data collection,analytics, and assurance, and may include the settings, protocols, andtables to monitor and analyze network infrastructure and endpointsconnected to the network. The network data platform 224 can collectmultiple types of information from network infrastructure devices,including syslog, SNMP, NetFlow, Switched Port Analyzer (SPAN), andstreaming telemetry, among others. The network data platform 224 canalso collect use contextual information shared from

In some embodiments, one or more Cisco DNA™ Center appliances canprovide the functionalities of the management layer 210, the networkcontrol platform 222, and the network data platform 224. The Cisco DNA™Center appliances can support horizontal scalability by addingadditional Cisco DNA™ Center nodes to an existing cluster; highavailability for both hardware components and software packages; backupand store mechanisms to support disaster discovery scenarios; role-basedaccess control mechanisms for differentiated access to users, devices,and things based on roles and scope; and programmable interfaces toenable integration with third party vendors. The Cisco DNA™ Centerappliances can also be cloud-tethered to provide for the upgrade ofexisting functions and additions of new packages and applicationswithout having to manually download and install them.

The AAA services 226 can provide identity and policy services for thenetwork layer 230 and physical layer 240, and may include the settings,protocols, and tables to support endpoint identification and policyenforcement services. The AAA services 226 can provide tools andworkflows to manage virtual networks and security groups, and to creategroup-based policies and contracts. The AAA services 226 can identifyand profile network infrastructure devices and endpoints usingAAA/RADIUS, 802.1X, MAC Authentication Bypass (MAB), web authentication,and EasyConnect, among others. The AAA services 226 can also collect anduse contextual information from the network control platform 222, thenetwork data platform 224, and the shared services 250, among others. Insome embodiments, Cisco® ISE can provide the AAA services 226.

The network layer 230 can be conceptualized as a composition of twolayers, an underlay 234 comprising physical and virtual networkinfrastructure (e.g., routers, switches, WLCs, etc.) and a Layer 3routing protocol for forwarding traffic, and an overlay 232 comprising avirtual topology for logically connecting wired and wireless users,devices, and things and applying services and policies to theseentities. Network elements of the underlay 234 can establishconnectivity between each other, such as via Internet Protocol (IP). Theunderlay may use any topology and routing protocol.

In some embodiments, the network controller 104 can provide a local areanetwork (LAN) automation service, such as implemented by Cisco DNA™Center LAN Automation, to automatically discover, provision, and deploynetwork devices. Once discovered, the automated underlay provisioningservice can leverage Plug and Play (PnP) to apply the required protocoland network address configurations to the physical networkinfrastructure. In some embodiments, the LAN automation service mayimplement the Intermediate System to Intermediate System (IS-IS)protocol. Some of the advantages of IS-IS include neighbor establishmentwithout IP protocol dependencies, peering capability using loopbackaddresses, and agnostic treatment of IPv4, IPv6, and non-IP traffic.

The overlay 232 can be a logical, virtualized topology built on top ofthe physical underlay 234, and can include a fabric data plane, a fabriccontrol plane, and a fabric policy plane. In some embodiments, thefabric data plane can be created via packet encapsulation using VirtualExtensible LAN (VXLAN) with Group Policy Option (GPO). Some of theadvantages of VXLAN-GPO include its support for both Layer 2 and Layer 3virtual topologies (overlays), and its ability to operate over any IPnetwork with built-in network segmentation.

In some embodiments, the fabric control plane can implement Locator/IDSeparation Protocol (LISP) for logically mapping and resolving users,devices, and things. LISP can simplify routing by removing the need foreach router to process every possible IP destination address and route.LISP can achieve this by moving remote destination to a centralized mapdatabase that allows each router to manage only its local routs andquery the map system to locate destination endpoints.

The fabric policy plane is where intent can be translated into networkpolicy. That is, the policy plane is where the network operator caninstantiate logical network policy based on services offered by thenetwork fabric 120, such as security segmentation services, quality ofservice (QoS), capture/copy services, application visibility services,and so forth.

Segmentation is a method or technology used to separate specific groupsof users or devices from other groups for the purpose of reducingcongestion, improving security, containing network problems, controllingaccess, and so forth. As discussed, the fabric data plane can implementVXLAN encapsulation to provide network segmentation by using the virtualnetwork identifier (VNI) and Scalable Group Tag (SGT) fields in packetheaders. The network fabric 120 can support both macro-segmentation andmicro-segmentation. Macro-segmentation logically separates a networktopology into smaller virtual networks by using a unique networkidentifier and separate forwarding tables. This can be instantiated as avirtual routing and forwarding (VRF) instance and referred to as avirtual network (VN). That is, a VN is a logical network instance withinthe network fabric 120 defined by a Layer 3 routing domain and canprovide both Layer 2 and Layer 3 services (using the VXLAN VNI toprovide both Layer 2 and Layer 3 segmentation). Micro-segmentationlogically separates user or device groups within a VN, by enforcingsource to destination access control permissions, such as by usingaccess control lists (ACLs). A scalable group is a logical objectidentifier assigned to a group of users, devices, or things in thenetwork fabric 120. It can be used as source and destination classifiersin Scalable Group ACLs (SGACLs). The SGT can be used to provideaddress-agnostic group-based policies.

In some embodiments, the fabric control plane node 110 may implement theLocator/Identifier Separation Protocol (LISP) to communicate with oneanother and with the management cloud 102. Thus, the control plane nodesmay operate a host tracking database, a map server, and a map resolver.The host tracking database can track the endpoints 130 connected to thenetwork fabric 120 and associate the endpoints to the fabric edge nodes126, thereby decoupling an endpoint's identifier (e.g., IP or MACaddress) from its location (e.g., closest router) in the network.

The physical layer 240 can comprise network infrastructure devices, suchas switches and routers 110, 122, 124, and 126 and wireless elements 108and 128 and network appliances, such as the network controllerappliance(s) 104, and the AAA appliance(s) 106.

The shared services layer 250 can provide an interface to externalnetwork services, such as cloud services 252; Domain Name System (DNS),DHCP, IP Address Management (IPAM), and other network address managementservices 254; firewall services 256; Network as a Sensor(Naas)/Encrypted Threat Analytics (ETA) services; and Virtual NetworkFunctions (VNFs) 260; among others. The management layer 202 and/or thecontroller layer 220 can share identity, policy, forwarding information,and so forth via the shared services layer 250 using APIs.

FIG. 4 illustrates an example of a physical topology for a multi-siteenterprise network 400. In this example, the network fabric comprisesfabric sites 420A and 420B. The fabric site 420A can include a fabriccontrol node 410A, fabric border nodes 422A and 422B, fabricintermediate nodes 424A and 424B (shown here in dashed line and notconnected to the fabric border nodes or the fabric edge nodes forsimplicity), and fabric edge nodes 426A-D. The fabric site 420B caninclude a fabric control node 410B, fabric border nodes 422C-E, fabricintermediate nodes 424C and 424D, and fabric edge nodes 426D-F. Multiplefabric sites corresponding to a single fabric, such as the networkfabric of FIG. 4, can be interconnected by a transit network. A transitnetwork can be a portion of a network fabric that has its own controlplane nodes and border nodes but does not have edge nodes. In addition,a transit network shares at least one border node with each fabric sitethat it interconnects.

In general, a transit network connects a network fabric to the externalworld. There are several approaches to external connectivity, such as atraditional IP network 436, traditional WAN 438A, Software-Defined WAN(SD-WAN) (not shown), or Software-Defined Access (SD-Access) 438B.Traffic across fabric sites, and to other types of sites, can use thecontrol plane and data plane of the transit network to provideconnectivity between these sites. A local border node can operate as thehandoff point from the fabric site, and the transit network can delivertraffic to other sites. The transit network may use additional features.For example, if the transit network is a WAN, then features likeperformance routing may also be used. To provide end-to-end policy andsegmentation, the transit network should be capable of carrying endpointcontext information (e.g., VRF, SGT) across the network. Otherwise, are-classification of the traffic may be needed at the destination siteborder.

The local control plane in a fabric site may only hold state relevant toendpoints that are connected to edge nodes within the local fabric site.The local control plane can register local endpoints via local edgenodes, as with a single fabric site (e.g., the network fabric 120). Anendpoint that isn't explicitly registered with the local control planemay be assumed to be reachable via border nodes connected to the transitnetwork. In some embodiments, the local control plane may not hold statefor endpoints attached to other fabric sites such that the border nodesdo not register information from the transit network. In this manner,the local control plane can be independent of other fabric sites, thusenhancing overall scalability of the network.

The control plane in the transit network can hold summary state for allfabric sites that it interconnects. This information can be registeredto the transit control plane by border from different fabric sites. Theborder nodes can register EID information from the local fabric siteinto the transit network control plane for summary EIDs only and thusfurther improve scalability.

The multi-site enterprise network 400 can also include a shared servicescloud 432. The shared services cloud 432 can comprise one or morenetwork controller appliances 404, one or more AAA appliances 406, andother shared servers (e.g., DNS; DHCP; IPAM; SNMP and other monitoringtools; NetFlow, syslog, and other data collectors, etc.) may reside.These shared services can generally reside outside of the network fabricand in a global routing table (GRT) of an existing network. In thiscase, some method of inter-VRF routing may be required. One option forinter-VRF routing is to use a fusion router, which can be an externalrouter that performs inter-VRF leaking (e.g., import/export of VRFroutes) to fuse the VRFs together. Multi-Protocol can be used for thisroute exchange since it can inherently prevent routing loops (e.g.,using the AS_PATH attribute). Other routing protocols can also be usedbut may require complex distribute-lists and prefix-lists to preventloops.

However, there can be several disadvantages in using a fusion router toachieve inter-VN communication, such as route duplication because routesleaked from one VRF to another are programmed in hardware tables and canresult in more TCAM utilization, manual configuration at multiple touchpoints wherever route-leaking is implemented, loss of SGT contextbecause SGTs may not be maintained across VRFs and must be re-classifiedonce the traffic enters the other VRF, and traffic hairpinning becausetraffic may need to be routed to the fusion router, and then back to thefabric border node.

SD-Access Extranet can provide a flexible and scalable method forachieving inter-VN communications by avoiding route duplication becauseinter-VN lookup occurs in the fabric control plane (e.g., software) suchthat route entries do not need to be duplicated in hardware; providing asingle touchpoint because the network management system (e.g., CiscoDNA™ Center) can automate the inter-VN lookup policy, making it a singlepoint of management; maintaining SGT context because the inter-VN lookupoccurs in the control plane node(s) (e.g., software), and avoidshair-pinning because inter-VN forwarding can occur at the fabric edge(e.g., the same intra-VN) so traffic does not need to hairpin at theborder node. Another advantage is that a separate VN can be made foreach of the common resources that are needed (e.g., a Shared ServicesVN, an Internet VN, a data center VN, etc.).

FIG. 5A depicts a block diagram of one example of an issue detection andvalidation system 500. Data can be collected from one or more userclients 501 to generate classifications, simulations, and issuevalidations by system 500. A data collection process 502 can collect thedata from the user clients 501 and provide it to an issue classificationengine 503 in a collated and structured format.

Issues classification engine 503 uses the collected data to learn issueclassifications (e.g., where a new event sequence or issue is provided),classify issues (e.g., via trained model, etc.), and/or provide eventsimulation data to a simulation engine 504 for reproducing an issue. Insome examples, issue classification engine 503 may include one or moretrained models for classifying, clustering, and further processing data.Additionally, issue classification engine 503 may provide an identifiedissue (e.g., via a classifier, etc.) to an issue validation process 506for validating issues identified by an assurance platform 505. As aresult, issue classification engine 503 may provide an external, toassurance platform 505, source of truth for improving accuracy androbustness of issue identifications.

Simulation engine 504 may include a client simulation process 508 and anissue simulation 509 to replicate event sequences (e.g., thoseclassified by issue classification engine 503) for processing byassurance platform 505 to generate issues (e.g., to send to issuevalidation process 506). Client simulation process 508 may replicatefeatures, such as operations and configurations, from various clientdevices, such as APs and WLCs, etc. In particular, client simulationprocess 508 can include a wireless client simulation process 510 and/ora sensor clients simulation process 512 for replicating features fromwireless client devices (e.g., MAC address, IP address, networkidentifiers, AP SSID, etc.) and client device sensor data (e.g., GPSdata, clock information, memory snapshots, etc.) respectively. Issuesimulation process 509 may include a device simulator 513 and a platforminterface 514. Device simulator 513 can treat (e.g., format according toa standard, derive additional information, etc.) received feature datafrom client simulation process 508 to provide to assurance platform 505via platform interface 514. Furthermore, and in some examples, devicesimulator 513 may provide additional data to assurance platform 505 fora simulated device such as backend or internal (to the network)information to reproduce a particular event sequence.

Assurance platform 505 can receive the simulated data to identify andgenerate issues (e.g., as discussed in regards to assurance functions212 above, etc.). Assurance platform 505 can include collectors 517 forretrieving data along the network, pipelines 516 for processingretrieved and received data, and issue generation engine 515 forgenerating issues reports. Generated issue reports may be sent to issuevalidation process 506 and issue classifications, provided issueclassification engine 503, may be used to validate the received issuereports.

FIG. 5B depicts a block diagram of an example issue classificationengine 521. In some examples, issue classification engine 521 may besubstantially the same as issue classification engine 503 above.Incoming event data 525 is provided to issue classification engine 521to classify and/or learn issues. Incoming event data 525 may includevarious data pulled from across an associated network and/or datarelated to respective endpoints (e.g., user clients 501, etc.).

Nevertheless, issue classification engine 521 may include an issueclassifier 523 for receiving incoming event data 525. In some examples,such as where issue classifier 523 is training an internal model (e.g.,a classifier, etc.), a historical event data repository 526 can provideone or more historical datasets to issue classifier 523 for trainingpurposes (e.g., to detect error distances and update the modelaccordingly, etc.). Further, issue classifier 523 may exchange data witha clustering service 524. Clustering service 524 identifies groupings(e.g., clusters) of classified issues and event sequences. In someexamples, clustering service 524 may provide input to issue classifier523 for training and/or as a feature fed to a model for classifyingissues (e.g., based on incoming event data 525, etc.).

Issue classifier 523 provides classified issues to a meta-languageconversion process 522. Meta-language conversion process 522 generates aformatted issue classification in a particular meta-languageinterpretable by downstream process. For example, meta-languageconversion process 522 may output an issue classification in thefollowing form: {timestamp, event-type, result-type, reason-code,packet-capture}. In this example, timestamp may include one or morevalues indicating the time the classified issue occurred, event-type mayinclude information describing the a final event or state related to theclassified issue (e.g., timeout type, etc.), reason-code may includereference information for a reason related to the event informationincluded in event-type, and packet-capture may include informationrelated to contents (e.g., origin information, destination information,device information, etc.) of the transmitted packet respective to theclassified issue. The generated meta-language issue classification maybe provided to downstream processes such as a simulation engine and/oran assurance platform.

FIGS. 5Ci and 5Cii are, respectively, a successful transaction eventsequence 530A and a failed transaction event sequence 530B. Successfultransaction event sequence 530A and failed transaction event sequence530B each depict an event sequence which may be processed by, forexample, system 500 to produce and validate issue classifications asdiscussed above. Both sequences 530A-B include a series of pairedclient-events 531 and reason-code 532 which are depicted here as rows ina chart for explanatory purposes only but which, in some examples, maybe 2-tuple data structures and the like. In particular, successfultransaction event sequence 531 includes a client-event and reason-codepair 533 indicating a successful device association (e.g., “RUN”). Ascan be seen, a “NO REASON” reason-code is provided as someclient-events, such as successful associations, do not entail areason-code to explain a successful association. In comparison, failedtransaction event sequence 530B includes a client-event and reason-codepair 534 indicating a failed device association (e.g., “AUTH_DONE” and“SPAM_REASSOC_TIMEOUT”). Here, the reason-code indicates that an eventrelated to concluding an authorization attempt occurred due to timeouterror (e.g., a response time, such as an authorization message returntime, took too long).

FIG. 5D depicts multiple ring graphs 540A-D displaying transactionanalytics. Transactions breakdown ring graph 540A depicts relativeproportions of transaction types across some networks. Portion 541Ashows 3.1% of transactions are excessive—meaning the transactionsuccessfully complete, however, it took an unacceptably larger amount oftime to complete (e.g., without timing out, such as in the case of atransaction failure). For example, where a transaction, such as a deviceassociation, takes 10 seconds or longer, the transaction may beidentified as an excess transaction. Portion 542A shows 16.9% oftransactions are failures (e.g., as discussed in regards to eventsequence 530B above) and portion 543A shows 80% of transactions aresuccesses (e.g., as discussed in regards to event sequence 530A above).In effect, issue classification engine 503 may detect the 3.1% excessivetransactions of portion 541A using a classifier and/or clustering andwithout having to handcraft a rules-based issue identifier for use by,for example, assurance platform 505.

Excessive transactions breakdown ring graph 540B depicts relativeproportions of excessive breakdowns (e.g., portion 541A) betweenmultiple device association attempts, single association attempts havingexcess authorization requests, and single association attempts havingexcess AP access attempts (e.g., device host configuration protocol(DHCP) message transmissions). Portion 541B shows 10.7% of excesstransactions include excess AP access attempts, portion 542B shows 20.9%of excess transactions include excess authorization attempts, andportion 543B shows 68.4% of excess transactions include multiple deviceassociation attempts. In some examples, multiple association attempts,excess authorization attempts, and/or excess AP access attempts may berepresented by a sequence of event codes. In effect, the sequence ofevent codes can be clustered to find similarities (e.g., groupings)based on subsequences and/or particular events within respectivesequences of event codes.

Multiple association excessive transaction breakdown ring graph 540Cdepicts relative proportions of excessive transactions due to multipleassociations (e.g., portion 543B) occurring at a single AP or acrossmultiple APs. In some examples, excess transactions over multiple APsmay indicate onboarding occurring concurrently to a device switchingradio channels and/or access points (e.g., base stations, wirelessrouters, etc.). Here, ring portion 541C shows 23% of multipleassociation excess transactions occur at a single AP and ring portion542C shows 77% of multiple association excess transactions occur overmultiple APs. In effect, excess transaction patterns can be identifiedby an issue classification engine based on additional information toevent sequences (e.g., such as information related to AP, WLC, etc.).

In comparison, excess authentication for single association requestsbreakdown ring graph 540D depicts relative proportions of excessivetransactions due to multiple authentication request events for a singledevice association request. Ring portion 541D shows that 49% of singleassociation requests resulting in excess authentication events include asingle additional excess authentication event. Ring portion 542D showsthat 28.6% of single association requests resulting in excessauthentication events include two additional excess authenticationevents. Ring portion 543D shows that 12.8% of single associationrequests resulting in excess authentication events include threeadditional excess authentication events. Ring portion 544D shows that9.59% of single association requests resulting in excess authenticationevents include four or more excess authentication events. In effect,various patterns, such as repetitions, can be identified by an issueclassification engine for identification (e.g., classification) ofissues.

FIGS. 5E-F illustrate a selection of particular event sequences whichmay be associated with excess transactions (e.g., “excess transactionpatterns”) by an issue classification engine. In particular, FIG. 5Edepicts a histogram 550A of an observed 10 most frequent, on one examplenetwork and without imputing limitation, excess transaction patterns.FIG. 5F depicts a frequency table 550B of an observed 15 most frequentexcess transaction patterns. Excess transaction patterns 551 a-o includeevent codes in a numerical format. As depicted by legend 555, event codenumerals may include, for example and without imputing limitation,ASSOC_START (“1”), ASSOC_DONE (“2”), AUTH_START (“3”), AUTH_DONE (“4”),4_WAY_KEY_DONE (“5”), MOBILITY_DONE (“6”), DHCP_DONE (“7”), RUN (“9”),and EAPOL_START (“12”). Accordingly, each respective excess transactionpattern includes a sequence of events. For example, excess transactionpattern 551 a include sequence “1-2-3-1-2-3-4-5-9” which indicates thefollowing event sequence: ASSOC_START, ASSOC_DONE, AUTH_START,ASSOC_START, ASSOC_DONE, AUTHR_START, AUTH_DONE, 4_WAY_KEY_DONE, RUN. Ineffect, the excess transaction includes two association event sequences(e.g., “ASSOC_START, ASSOC_DONE, AUTH_START”), such as where a deviceattempts to connect to an AP multiple times before being authorized.

Histogram 550A depicts a bar view of excess transaction patterns againsta percentage 559. In comparison, frequency table 550B depicts a tabularview of excess transaction patterns split across a transaction patterncolumn 556, including event sequences along respective rows, andrespectively paired frequency percentages in a frequency column 557.

FIG. 5G depicts a method 560 for validating issues identificationsproduced by an assurance platform (e.g., assurance platform 505). Insome examples, method 560 can be performed by system 500 discussedabove.

At step 561, client onboarding attempt information is received. Clienton-boarding attempt may be real time or may be historical data used totrain classifier and/or clustering processes.

At step 562, events and corresponding reason codes are identified fromthe onboarding attempt information for clients that have failed tosuccessfully onboard. Event and reason codes may be included in clientonboarding attempt information. In some examples, an upstream processmay package event and reason codes into the information.

At step 563, additional context information for the failed attempts isidentified and associated with the respective attempts. Additionalcontext information may include WLC and/or AP information such as IPaddresses, SSID information, and the like as well as user clientspecific information such as location, etc.

At step 564, issue groups are generated based on the events, reasoncodes, and context information. Issue groups can be generated using aclassifier and/or clustering process. The classifier and/or clusteringprocess may identify shared event sequences and context informationbetween reason codes.

At step 565, similar event patterns are aggregated into shared issuegroups. Issue groups can be further reduced to shared issue groupincluding partially shared event sequences between issue groups. Forexample, where a particular event sequence is identified as closelyrelated to a particular reason code, and the particular event sequenceoccurs across multiple issue groups as a subsequence, the multiple issuegroups can be combined into a shared issue group.

At step 566, new client onboarding information is received. In someexamples, the new client onboarding information may be provided backinto classifier and clustering training steps (e.g., steps 561-565) aswell as be processed for downstream issue validation.

At step 567, a first issue is identified based on the shared group andnew client onboarding information. The first issue may be identifiedusing a classification engine (e.g., issue classification engine 503).In some examples, the classification engine can include a trained model(e.g., trained via steps 561-565 above) for classifying event sequencesas particular issues. Additionally, in some examples, the classificationengine can instead or also include a trained model for clustering eventsequences and generating issue groups and/or shared issue groups (asdiscussed above).

At step 568, events and context information are simulated related to thenew client onboarding information to generate a second issueidentification by an assurance platform. The events and contextinformation can be provided to a simulation engine (e.g., simulationengine 504 discussed above) to reproduce a simulated occurrencesubstantially similar to an occurrence from which the new clientonboarding information was produced.

At step 569, the second issue identification is validated with the firstissue identification. In effect, the assurance platform issueidentification can be checked against the classification issueidentification. In some examples, one or the other may be treated as asource of truth and so, based on which is the source of truth and thevalidation, issue classification engine can adjust (e.g., update a modelparameters to improve issue classification) or the assurance platformmay be updated (e.g., rules-based processes may be adjusted) to coincidethe second issue identification with the first issue identification.

FIG. 6A and FIG. 6B illustrate systems in accordance with variousembodiments. The more appropriate system will be apparent to those ofordinary skill in the art when practicing the various embodiments.Persons of ordinary skill in the art will also readily appreciate thatother systems are possible.

FIG. 6A illustrates an example of a bus computing system 600 wherein thecomponents of the system are in electrical communication with each otherusing a bus 605. The computing system 600 can include a processing unit(CPU or processor) 610 and a system bus 605 that may couple varioussystem components including the system memory 615, such as read onlymemory (ROM) 620 and random access memory (RAM) 625, to the processor610. The computing system 600 can include a cache 612 of high-speedmemory connected directly with, in close proximity to, or integrated aspart of the processor 610. The computing system 600 can copy data fromthe memory 615, ROM 620, RAM 625, and/or storage device 630 to the cache612 for quick access by the processor 610. In this way, the cache 612can provide a performance boost that avoids processor delays whilewaiting for data. These and other modules can control the processor 610to perform various actions. Other system memory 615 may be available foruse as well. The memory 615 can include multiple different types ofmemory with different performance characteristics. The processor 610 caninclude any general purpose processor and a hardware module or softwaremodule, such as module 1 632, module 2 634, and module 3 636 stored inthe storage device 630, configured to control the processor 610 as wellas a special-purpose processor where software instructions areincorporated into the actual processor design. The processor 610 mayessentially be a completely self-contained computing system, containingmultiple cores or processors, a bus, memory controller, cache, etc. Amulti-core processor may be symmetric or asymmetric.

To enable user interaction with the computing system 600, an inputdevice 645 can represent any number of input mechanisms, such as amicrophone for speech, a touch-protected screen for gesture or graphicalinput, keyboard, mouse, motion input, speech and so forth. An outputdevice 635 can also be one or more of a number of output mechanismsknown to those of skill in the art. In some instances, multimodalsystems can enable a user to provide multiple types of input tocommunicate with the computing system 600. The communications interface640 can govern and manage the user input and system output. There may beno restriction on operating on any particular hardware arrangement andtherefore the basic features here may easily be substituted for improvedhardware or firmware arrangements as they are developed.

The storage device 630 can be a non-volatile memory and can be a harddisk or other types of computer readable media which can store data thatare accessible by a computer, such as magnetic cassettes, flash memorycards, solid state memory devices, digital versatile disks, cartridges,random access memory, read only memory, and hybrids thereof.

As discussed above, the storage device 630 can include the softwaremodules 632, 634, 636 for controlling the processor 610. Other hardwareor software modules are contemplated. The storage device 630 can beconnected to the system bus 605. In some embodiments, a hardware modulethat performs a particular function can include a software componentstored in a computer-readable medium in connection with the necessaryhardware components, such as the processor 610, bus 605, output device635, and so forth, to carry out the function.

FIG. 6B illustrates an example architecture for a chipset computingsystem 650 that can be used in accordance with an embodiment. Thecomputing system 650 can include a processor 655, representative of anynumber of physically and/or logically distinct resources capable ofexecuting software, firmware, and hardware configured to performidentified computations. The processor 655 can communicate with achipset 660 that can control input to and output from the processor 655.In this example, the chipset 660 can output information to an outputdevice 665, such as a display, and can read and write information tostorage device 670, which can include magnetic media, solid state media,and other suitable storage media. The chipset 660 can also read datafrom and write data to RAM 675. A bridge 680 for interfacing with avariety of user interface components 685 can be provided for interfacingwith the chipset 660. The user interface components 685 can include akeyboard, a microphone, touch detection and processing circuitry, apointing device, such as a mouse, and so on. Inputs to the computingsystem 650 can come from any of a variety of sources, machine generatedand/or human generated.

The chipset 660 can also interface with one or more communicationinterfaces 690 that can have different physical interfaces. Thecommunication interfaces 690 can include interfaces for wired andwireless LANs, for broadband wireless networks, as well as personal areanetworks. Some applications of the methods for generating, displaying,and using the technology disclosed herein can include receiving ordereddatasets over the physical interface or be generated by the machineitself by the processor 655 analyzing data stored in the storage device670 or the RAM 675. Further, the computing system 650 can receive inputsfrom a user via the user interface components 685 and executeappropriate functions, such as browsing functions by interpreting theseinputs using the processor 655.

It will be appreciated that computing systems 600 and 650 can have morethan one processor 610 and 655, respectively, or be part of a group orcluster of computing devices networked together to provide greaterprocessing capability.

For clarity of explanation, in some instances the various embodimentsmay be presented as including individual functional blocks includingfunctional blocks comprising devices, device components, steps orroutines in a method embodied in software, or combinations of hardwareand software.

In some embodiments the computer-readable storage devices, mediums, andmemories can include a cable or wireless signal containing a bit streamand the like. However, when mentioned, non-transitory computer-readablestorage media expressly exclude media such as energy, carrier signals,electromagnetic waves, and signals per se.

Methods according to the above-described examples can be implementedusing computer-executable instructions that are stored or otherwiseavailable from computer readable media. Such instructions can comprise,for example, instructions and data which cause or otherwise configure ageneral purpose computer, special purpose computer, or special purposeprocessing device to perform a certain function or group of functions.Portions of computer resources used can be accessible over a network.The computer executable instructions may be, for example, binaries,intermediate format instructions such as assembly language, firmware, orsource code. Examples of computer-readable media that may be used tostore instructions, information used, and/or information created duringmethods according to described examples include magnetic or opticaldisks, flash memory, USB devices provided with non-volatile memory,networked storage devices, and so on.

Devices implementing methods according to these disclosures can comprisehardware, firmware and/or software, and can take any of a variety ofform factors. Some examples of such form factors include general purposecomputing devices such as servers, rack mount devices, desktopcomputers, laptop computers, and so on, or general purpose mobilecomputing devices, such as tablet computers, smart phones, personaldigital assistants, wearable devices, and so on. Functionality describedherein also can be embodied in peripherals or add-in cards. Suchfunctionality can also be implemented on a circuit board among differentchips or different processes executing in a single device, by way offurther example.

The instructions, media for conveying such instructions, computingresources for executing them, and other structures for supporting suchcomputing resources are means for providing the functions described inthese disclosures.

Examples are provided herein to enhance understanding of the disclosure.A specific set of statements are provided as follows:

Statement 1: A computer-implemented method includes receiving networktraffic data, identifying event sequences within the network trafficdata, generating definition groups by performing clustering on theidentified event sequences, the definition groups including eventsequence characteristics associated with one or more network issues,providing one or more of the event sequences to an issue simulationengine to simulate the one or more of the event sequences and generatinga first issue identification by an assurance platform, generating asecond issue identification based on the definition groups and the oneor more of the event sequences, and validating the first issueidentification with the second issue identification.

Statement 2: A method according to Statement 1 includes the issuesimulation engine including a client simulation for simulating a portionof the provided one or more of the event sequences from a client deviceperspective.

Statement 3: A method according to any of the preceding Statementsincludes the definition groups associated with onboarding failures.

Statement 4: A method according to Statement 3 includes the onboardingfailures including an excess onboarding transaction.

Statement 5: A method according to any of the preceding Statementsincludes the network traffic data being generated by network devicesincluding one of a wireless local area network controller (WLC) or anaccess point (AP).

Statement 6: A method according to any of the preceding Statementsincludes the network traffic data including a data model including oneof a wireless client state, client statistics, AP radio frequency (RF)statistics, an over air packet capture, a network services keyperformance indicator (KPI), or a device health KPI.

Statement 7: A method according to any of the preceding Statementsincludes generating the first issue identification including applying,by the assurance platform, a rules-based issue identification process tothe simulated event sequences.

Statement 8: A system includes one or more processors, and a memorystoring instructions to receive network traffic data, identify eventsequences within the network traffic data, generate definition groups byperforming clustering on the identified event sequences, the definitiongroups including event sequence characteristics associated with one ormore network issues, provide one or more of the event sequences to anissue simulation engine to simulate the one or more of the eventsequences and generating a first issue identification by an assuranceplatform, generate a second issue identification based on the definitiongroups and the one or more of the event sequences, and validate thefirst issue identification with the second issue identification.

Statement 9: A non-transitory computer readable medium storesinstructions that, when executed by one or more processors, cause theone or more processors to receive network traffic data, identify eventsequences within the network traffic data, generate definition groups byperforming clustering on the identified event sequences, the definitiongroups including event sequence characteristics associated with one ormore network issues, provide one or more of the event sequences to anissue simulation engine to simulate the one or more of the eventsequences and generating a first issue identification by an assuranceplatform, generate a second issue identification based on the definitiongroups and the one or more of the event sequences, and validate thefirst issue identification with the second issue identification.

Although a variety of examples and other information was used to explainaspects within the scope of the appended claims, no limitation of theclaims should be implied based on particular features or arrangements insuch examples, as one of ordinary skill would be able to use theseexamples to derive a wide variety of implementations. Further andalthough some subject matter may have been described in languagespecific to examples of structural features and/or method steps, it isto be understood that the subject matter defined in the appended claimsis not necessarily limited to these described features or acts. Forexample, such functionality can be distributed differently or performedin components other than those identified herein. Rather, the describedfeatures and steps are disclosed as examples of components of systemsand methods within the scope of the appended claims.

What is claimed is:
 1. A computer-implemented method comprising:receiving network traffic data; identifying event sequences within thenetwork traffic data; generating definition groups by performingclustering on the identified event sequences, the definition groupscomprising event sequence characteristics associated with one or morenetwork issues; first simulating the identified event sequences in anissue simulation engine to simulate the one or more of the eventsequences; generating, based on results of the first simulating, a firstissue identification by an assurance platform; second simulating a mostrecent one of the identified event sequences; generating, based onresults of the second simulating and the generated definition groups, asecond issue identification; and validating the first issueidentification with the second issue identification.
 2. The method ofclaim 1, wherein the issue simulation engine includes a clientsimulation for simulating a portion of the event sequences from a clientdevice perspective.
 3. The method of claim 1, wherein the definitiongroups are associated with onboarding failures.
 4. The method of claim3, wherein the onboarding failures comprise an excess onboardingtransaction.
 5. The method of claim 1, wherein the network traffic datais generated by network devices comprising one of a wireless local areanetwork controller (WLC) or an access point (AP).
 6. The method of claim1, wherein the network traffic data comprises a data model including oneof a wireless client state, client statistics, AP radio frequency (RF)statistics, an over air packet capture, a network services keyperformance indicator (KPI), or a device health KPI.
 7. The method ofclaim 1, wherein generating the first issue identification comprisesapplying, by the assurance platform, a rules-based issue identificationprocess to the simulated event sequences.
 8. A system comprising: one ormore processors; and a memory comprising instructions to: receivenetwork traffic data; identify event sequences within the networktraffic data; first simulate the identified event sequences; generate,based on the results of the first simulate, definition groups byperforming clustering on the identified event sequences, the definitiongroups comprising event sequence characteristics associated with one ormore network issues; second simulate a most recent one of the identifiedevent sequences in an issue simulation engine to simulate the one ormore of the event sequences and generating a first issue identificationby an assurance platform; generate, based on the results of the secondsimulate and the generated definition groups, a second issueidentification based on the definition groups and the one or more of theevent sequences; and validate the first issue identification with thesecond issue identification.
 9. The system of claim 8, wherein the issuesimulation engine includes a client simulation for simulating a portionof the event sequences from a client device perspective.
 10. The systemof claim 8, wherein the definition groups are associated with onboardingfailures.
 11. The system of claim 10, wherein the onboarding failurescomprise an excess onboarding transaction.
 12. The system of claim 8,wherein the network traffic data is generated by network devicescomprising one of a wireless local area network controller (WLC) or anaccess point (AP).
 13. The system of claim 8, wherein the networktraffic data comprises a data model including one of a wireless clientstate, client statistics, AP radio frequency (RF) statistics, an overair packet capture, a network services key performance indicator (KPI),or a device health KPI.
 14. The system of claim 8, wherein generatingthe first issue identification comprises applying, by the assuranceplatform, a rules-based issue identification process to the simulatedevent sequences.
 15. A non-transitory computer readable mediumcomprising instructions that, when executed by one or more processors,cause the one or more processors to: receive network traffic data;identify event sequences within the network traffic data; first simulatethe identified event sequences; generate, based on the results of thefirst simulate, definition groups by performing clustering on theidentified event sequences, the definition groups comprising eventsequence characteristics associated with one or more network issues;second simulating a most recent one of the identified event sequences inan issue simulation engine to simulate the one or more of the eventsequences and generating a first issue identification by an assuranceplatform; generate, based on the results of the second simulate and thegenerated definition groups, a second issue identification based on thedefinition groups and the one or more of the event sequences; andvalidate the first issue identification with the second issueidentification.
 16. The non-transitory computer readable medium of claim15, wherein the issue simulation engine includes a client simulation forsimulating a portion of the event sequences from a client deviceperspective.
 17. The non-transitory computer readable medium of claim15, wherein the definition groups are associated with onboardingfailures each comprising an excess on-boarding transaction.
 18. Thenon-transitory computer readable medium of claim 15, wherein the networktraffic data is generated by network devices comprising one of awireless local area network controller (WLC) or an access point (AP).19. The non-transitory computer readable medium of claim 15, wherein thenetwork traffic data comprises a data model including one of a wirelessclient state, client statistics, AP radio frequency (RF) statistics, anover air packet capture, a network services key performance indicator(KPI), or a device health KPI.
 20. The non-transitory computer readablemedium of claim 15, wherein generating the first issue identificationcomprises applying, by the assurance platform, a rules-based issueidentification process to the simulated event sequences.